机器学习和认知科学的最新工作表明，了解因果信息对于智力的发展至关重要。使用``Blicket otter''环境的认知科学的广泛文献表明，孩子们擅长多种因果推理和学习。我们建议将该环境适应机器​​学习代理。当前机器学习算法的关键挑战之一是建模和理解因果关系：关于因果关系集的可转移抽象假设。相比之下，即使是幼儿也会自发学习和使用因果关系。在这项工作中，我们提出了一个新的基准 - 一种灵活的环境，可以评估可变因果溢出物下的现有技术 - 并证明许多现有的最新方法在这种环境中概括了困难。该基准的代码和资源可在https://github.com/cannylab/casual_overhypothess上获得。

In optimization-based approaches to inverse problems and to statistical estimation, it is common to augment the objective with a regularizer to address challenges associated with ill-posedness. The choice of a suitable regularizer is typically driven by prior domain information and computational considerations. Convex regularizers are attractive as they are endowed with certificates of optimality as well as the toolkit of convex analysis, but exhibit a computational scaling that makes them ill-suited beyond moderate-sized problem instances. On the other hand, nonconvex regularizers can often be deployed at scale, but do not enjoy the certification properties associated with convex regularizers. In this paper, we seek a systematic understanding of the power and the limitations of convex regularization by investigating the following questions: Given a distribution, what are the optimal regularizers, both convex and nonconvex, for data drawn from the distribution? What properties of a data source govern whether it is amenable to convex regularization? We address these questions for the class of continuous and positively homogenous regularizers for which convex and nonconvex regularizers correspond, respectively, to convex bodies and star bodies. By leveraging dual Brunn-Minkowski theory, we show that a radial function derived from a data distribution is the key quantity for identifying optimal regularizers and for assessing the amenability of a data source to convex regularization. Using tools such as $\Gamma$-convergence, we show that our results are robust in the sense that the optimal regularizers for a sample drawn from a distribution converge to their population counterparts as the sample size grows large. Finally, we give generalization guarantees that recover previous results for polyhedral regularizers (i.e., dictionary learning) and lead to new ones for semidefinite regularizers.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

我们通过与与前面令牌的局部相似度，通过调节从大语料库检索的文档块来增强自动回归语言模型。尽管使用25美元\时分，我们的检索增强型变压器（RetroCro）的检索增强型变压器（RetroCr）对GPT-3和侏罗纪-1获得了可比性的性能。微调后，复古表演转换为下游知识密集型任务，如问题应答。复古结合了冷冻BERT猎犬，一种可微分的编码器和块状的横向机制，以预测基于数量级的令牌，而不是训练期间通常消耗的数量。我们通常从头开始训练复古，还可以快速改造预先接受的变压器，通过检索，仍然达到良好的性能。我们的工作通过以前所未有的规模开辟了通过显式内存改进语言模型的新途径。

虽然通过简单的因素问题回答，文本理解的大量进展，但更加全面理解话语仍然存在重大挑战。批判性地反映出文本的人将造成好奇心驱动，通常是开放的问题，这反映了对内容的深刻理解，并要求复杂的推理来回答。建立和评估这种类型的话语理解模型的关键挑战是缺乏注释数据，特别是因为找到了这些问题的答案（可能根本不回答），需要高度的注释载荷的高认知负荷。本文提出了一种新的范式，使可扩展的数据收集能够针对新闻文件的理解，通过话语镜头查看这些问题。由此产生的语料库DCQA（疑问回答的话语理解）包括在607名英语文件中的22,430个问题答案对组成。 DCQA以自由形式，开放式问题的形式捕获句子之间的话语和语义链接。在评估集中，我们向问题上的问题提交了来自好奇数据集的问题，我们表明DCQA提供了有价值的监督，以回答开放式问题。我们还在使用现有的问答资源设计预训练方法，并使用合成数据来适应不可批售的问题。

最近已被证明大型语言模型在各种任务集中获得合理的零射普通化（Brown等，2020）。它已经假设这是语言模型的隐式多任务学习的结果，在语言模型中的预押（Radford等，2019）。可以通过明确的多任务学习直接引起零拍常规化？为了以缩放测试这个问题，我们开发一个系统，以便轻松地将任何自然语言任务映射到人类可读的提示表单中。我们转换一组大量的监督数据集，每个数据集都有多个提示，具有不同的措辞。这些提示的数据集允许基准测试模型执行完全看不见的任务的能力。我们介绍了一个普拉克尔编码器 - 解码器模型（Raffel等，2020; Lester等，2021），覆盖各种任务。该模型在多个标准数据集中达到强大的零点性能，通常优于其尺寸的型号超过16倍。此外，我们的方法对来自Big-替补基准测试的任务子集具有强烈性能，优于其尺寸的6倍。所有提示和培训的型号都可以在https://github.com/ bigscience-workshop / protectsource / httpsource / https：//huggingface.co/bigscience/t0pp。